1. Field of the Invention
The present invention relates to an image processing technique to increase definition of an image produced by an image pickup system.
2. Description of the Related Art
As a technique to increase definition of an image produced by an image pickup system, a super-resolution technique is known. In such a super-resolution technique, a reconstruction-based super-resolution technique performs modeling of a deterioration process in the image pickup system to produce an observation model, and solves an inverse problem thereof to increase the definition.
There are proposed methods for solving the inverse problem such as a frequency-domain technique that uses aliasing on a captured image and a MAP (Maximum a Posteriori) method that optimizes a posteriori probability of prior information to estimate a high-definition image. Moreover, a POCS (Projection onto Convex Sets) method is also proposed which combines various prior information to estimate a high-definition image.
The observation model is constituted by a response function of the image pickup system and samplings in an image sensor, the response function being obtained from a motion of the image pickup system, a deterioration in an optical system and effects of a CF (Color Filter) and an LPF (Low-Pass Filter). It is known that accuracy of reproducibility of the observation model with respect to the deterioration process in an actual image pickup system significantly influences accuracy of increasing the definition obtained by the super-resolution technique, and thus various observation models are proposed.
Japanese Patent Laid-Open No. 2005-095328 discloses a super-resolution process that uses an observation model utilizing, as the response function of the image pickup system, a PSF (Point Spread Function) of an optical system showing deterioration in the optical system.
However, the super-resolution process disclosed in Japanese Patent Laid-Open No. 2005-095328 uses a same response function for the entire image, and therefore cannot sufficiently increase the definition in an image area where the response function to be used is significantly different from an actual response function of the image pickup system.
On the other hand, using different response functions for respective pixels constituting an entire image in the super-resolution process makes it possible to solve such a problem. However, a memory capacity required for storing the response functions for all the respective pixels becomes large, which it is undesirable.